Deep Challenges of Natural Vision for AI

Much research on vision assumes that the functions of vision are obvious and attempts to investigate, or build, mechanisms that those functions require. However, there are many functions of biological vision that go unnoticed, or are mis-described, and that holds back both science and engineering. Some generally unrecognized or poorly characterised functions of vision are described, including uses of vision to provide information about what is not the case, the variety of requirements for integrating visual input across time and space, and the roles of vision in mathematical discoveries leading up to Euclid’s Elements, for instance reasoning about what is and is not possible in the environment and why. Some visual functions are important for “online intelligence”, e.g. in visual servo-control, others for “offline intelligence”, e.g. in planning, designing or explaining. Vision can be important in “social intelligence” including requesting or providing shareable information, and acquiring and using information about other individuals (e.g. what they can or cannot reach, or do, or see, or their states of mind). Exhaustively specifying functions of vision is impossible, since they can change. However, we can start with a set of unnoticed requirements relevant to research on vision in intelligent animals and machines.